Skip to content

[DX-2884, DX-2901, DX-2911, DX-3052] hybrid local cre env#21318

Draft
Tofel wants to merge 39 commits intodevelopfrom
dx-3040-remote-chip-sink
Draft

[DX-2884, DX-2901, DX-2911, DX-3052] hybrid local cre env#21318
Tofel wants to merge 39 commits intodevelopfrom
dx-3040-remote-chip-sink

Conversation

@Tofel
Copy link
Contributor

@Tofel Tofel commented Feb 26, 2026

Summary

This PR delivers the full CRE hybrid environment: components can run locally (Docker) or remotely (remote Docker agent on EC2), including mixed-mode topologies.
Beyond feature enablement, it also hardens and simplifies the architecture by removing obsolete transport paths, consolidating remote execution code, improving runtime/config resolution, and expanding test coverage.

Scope of changes

  • Hybrid execution model

    • Introduced placement-aware orchestration (local / remote) for blockchains, JD, and nodesets.
    • Added remote component lifecycle management through a dedicated remote agent.
    • Enabled mixed deployments where local and remote components interoperate in one CRE setup.
  • Remote execution architecture

    • Consolidated remote execution logic under a dedicated remoteexec package tree (agent + client + orchestration helpers).
    • Added descriptor-driven remote startup flow to reduce repeated orchestration code and improve cohesion.
    • Simplified remote runtime resolution and client initialization flow.
  • Networking and mixed-mode correctness

    • Implemented/normalized host and endpoint rewriting for cross-placement communication.
    • Improved P2P/bootstrapping and gateway URL handling for local<->remote paths.
    • Added fixture relay capabilities critical for remote->local connectivity in tests.
    • Fixed reconstructed-state gateway incoming host behavior for mixed mode.
  • Remote agent operations and debuggability

    • Added agent introspection endpoints and client support:
      • status
      • lock visibility
      • component log retrieval
    • Improved relay visibility with richer relay metadata for troubleshooting.
    • Added compatibility/versioning guardrails (capability/protocol oriented).
  • Simplification and cleanup

    • Removed SSM mode and related tunnel-era abstractions.
    • Removed local-agent legacy assumptions and dead code paths.
    • Centralized execution planning/normalization and address rewrite helpers.
    • Removed no-op/unused pieces and reduced cross-file coupling.
  • Configuration and naming cleanup

    • Renamed EC2-centric env vars to generic remote naming:
      • CRE_REMOTE_AGENT_URL
      • CRE_REMOTE_AGENT_PORT
      • CRE_REMOTE_HOST_IP
      • CRE_REMOTE_AGENT_EC2_INSTANCE_ID
    • Removed CRE_AWS_PROFILE; standard AWS env resolution applies where EC2 discovery is used.
    • Ensured non-EC2 explicit remote URL/host paths do not require AWS-related env vars.
  • Docs and UX

    • Updated runbooks and hybrid troubleshooting documentation.
    • Added/updated remote debugging skills/docs.
    • Added topology/placement visualization improvements for remote-aware startup context.

Testing

  • Expanded unit coverage for:
    • remote client/runtime resolution
    • remote stop/artifact flows
    • agent server handlers/state/logs/relay logic
    • environment placement planning and normalization
    • DON config and mixed-mode address resolution
  • Standardized assertions on testify/require.
  • Added high/medium ROI tests for critical relay and deployment seams.
  • Validated with package-level tests, lint gates, and mixed-mode e2e smoke runs.

Outcome

The resulting hybrid CRE environment is functional, more maintainable, and easier to debug: remote execution is cohesive, mixed-mode behavior is explicit and tested, and legacy complexity has been removed.

--- ALSO:

Adds standalone CHiP test sink support across local CLI and remote agent flows, so sink lifecycle and event visibility can be managed consistently in mixed/local/remote CRE setups.

Summary

  • Adds env beholder sink command wiring and standalone sink lifecycle handling in CLI (start/stop/status/events).
  • Introduces remote agent CHiP sink lifecycle APIs and runtime/status integration.
  • Adds remote client wrappers for CHiP sink operations.
  • Adds shared CHiP event decoding logic (event_decode) and reusable sink server package for runtime usage.
  • Persists remote agent runtime state needed for follow-up sink operations.
  • Updates test helpers and adds unit/handler tests for sink lifecycle + remote client behavior.

Tofel added 30 commits February 16, 2026 17:06
# Conflicts:
#	core/scripts/go.mod
#	core/scripts/go.sum
#	system-tests/lib/cre/contracts/keystone_test.go
#	system-tests/lib/cre/don.go
#	system-tests/lib/go.sum
#	system-tests/tests/go.sum
#	system-tests/tests/smoke/cre/cre_suite_test.go
@github-actions
Copy link
Contributor

github-actions bot commented Feb 26, 2026

✅ No conflicts with other open PRs targeting develop

@Tofel Tofel changed the title WIP of standalone chip ingress sink [DX-2884] Remote CHiP Ingress test sink Feb 26, 2026
@trunk-io
Copy link

trunk-io bot commented Feb 26, 2026

Static BadgeStatic BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

@github-actions
Copy link
Contributor

github-actions bot commented Mar 3, 2026

CORA - Analysis Skipped

Reason: The number of code owners (2) is less than the minimum required (5) and/or the number of CODEOWNERS entries with changed files (3) is less than the minimum required (2).

@github-actions
Copy link
Contributor

github-actions bot commented Mar 3, 2026

I see you added a changeset file but it does not contain a tag. Please edit the text include at least one of the following tags:

  • #added For any new functionality added.
  • #breaking_change For any functionality that requires manual action for the node to boot.
  • #bugfix For bug fixes.
  • #changed For any change to the existing functionality.
  • #db_update For any feature that introduces updates to database schema.
  • #deprecation_notice For any upcoming deprecation functionality.
  • #internal For changesets that need to be excluded from the final changelog.
  • #nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
  • #removed For any functionality/config that is removed.
  • #updated For any functionality that is updated.
  • #wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

@Tofel Tofel changed the title [DX-2884] Remote CHiP Ingress test sink [DX-2884, DX-2901, DX-2911] hybrid local cre env Mar 3, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Mar 3, 2026

I see you updated files related to core. Please run make gocs in the root directory to add a changeset as well as in the text include at least one of the following tags:

  • #added For any new functionality added.
  • #breaking_change For any functionality that requires manual action for the node to boot.
  • #bugfix For bug fixes.
  • #changed For any change to the existing functionality.
  • #db_update For any feature that introduces updates to database schema.
  • #deprecation_notice For any upcoming deprecation functionality.
  • #internal For changesets that need to be excluded from the final changelog.
  • #nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
  • #removed For any functionality/config that is removed.
  • #updated For any functionality that is updated.
  • #wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

@Tofel Tofel changed the title [DX-2884, DX-2901, DX-2911] hybrid local cre env [DX-2884, DX-2901, DX-2911, DX-3052] hybrid local cre env Mar 3, 2026
@cl-sonarqube-production
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant